A conventional camera sensor employs filters that pass through red, green, and blue light to different pixels on a sensor. The color measured on the sensor (Bayer values) is not the final color representation used in the picture, as a conventional camera will have the following steps: (i) black level subtraction, (ii) white balance, (iii) demosaic (generating RGB triples for each pixel from the sensor Bayer Color Filter Array (CFA) pattern), (iv) color correction (i.e., a three-dimensional mapping from an input RGB color space to an output RGB color space), (v) tone curve (mapping the color from a linear camera space to a nonlinear output space—linear means that the amount of light represented by a sample is directly in proportion to a corresponding digital value) and (vi) conversion from RGB to YUV.
Color correction is commonly used in digital cameras because a spectral response of the camera photo-receptors does not match a desired response in an output color space. The color correction is used to produce a picture that has accurate and aesthetically pleasing colors. Unfortunately, the amplitude of noise, especially in some colors, can be greatly increased by the color correction.
Several conventional color correction methods are currently available. Some methods use a matrix (M) multiplication to calculate an RGB output vector from a red, green, blue (RGB) input vector, such as:R_out=M11×R_in+M12×G_in+M13×B_inG_out=M21×R_in+M22×G_in+M23×B_inB_out=M31×R_in+M32×G_in+M33×B_inFor example, the Adobe “Digital Negative (DNG) Specifications” file format specifies color correction by means of a matrix. Other conventional color correction methods use a three-dimensional lookup table, with interpolation between the table entries. For example, U.S. Pat. No. 4,275,413 describes a method for tetrahedral interpolation.
Referring to FIG. 1, a block diagram of a conventional camera color processing pipeline 20 is shown. Noise reduction is conventionally done at a few of many different points in the pipeline 20. FIG. 1 shown the six color processing steps mentioned above plus a YUV to YUV color correction step, a compression step and several noise reduction steps. A typical camera performs noise reduction only in some of the places illustrated.
Different tradeoffs exist between implementing the noise reduction at different points in the pipeline 20. From a cost point of view, noise reduction is less expensive to implement: (i) before demosaicing as opposed to after demosaicing, (ii) after tone correction as opposed to before tone correction and (iii) after conversion to YUV as opposed to just before conversion to YUV. After demosaicing the data rate is tripled. After tone correction fewer bits per sample typically exist. After conversion to YUV, especially if down-sampling to the 4:2:2 format or the 4:2:0 format is involved, the sample rate is often reduced.
From a quality point of view (better quality meaning more noise reduction with fewer or less-objectionable artifacts such as loss of sharpness and detail) filtering at different places has different advantages. For example, filtering early in the processing can be advantageous because noise has not yet spread between the samples or significantly changed. On the other hand, filtering after conversion to YUV can be advantageous because the type and amount of filtering suitable for chrominance data are different from the type and amount of filtering suitable for luminance data. Typically, the chrominance data is filtered more heavily than the luminance data.
Referring to FIG. 2, a diagram of an example signal amplitude before 22 and after 24 a conventional spatial noise filtering is shown. Noise reduction by spatial noise filtering distinguishes between signal variations that are due to noise versus signal variations that are due to changes in the underlying signal. Different conventional techniques judge (i) small signal variations as noise to be reduced or eliminated and (ii) large signal variations where the underlying signal is to be maintained.
Referring to FIG. 3, a diagram of an example signal amplitude before 26 and after 28 a conventional color correction is shown. Typically, the amplitude of an observed variation relative to the noise standard deviation is used to determine if the variation is due to noise or variation in the underlying signal. As described earlier, color correction can increase noise in some colors. In the diagram, some noise 30 is not increased by the color correction while other noise 32 is increased.
Due to the color-dependent noise amplification, a determination or even approximation of the noise standard deviation after color correction can be difficult or expensive, especially if the color correction process is complex. Therefore, the quality of any noise reduction after color correction is usually degraded, for example, noise reduction done after conversion to YUV. Noise reduction done in YUV may be appropriate for the reasons described above (i.e., cost and ability to control type and amount of filtering for chrominance independent of luminance). However, the effectiveness of such noise reductions can be degraded by occurring after the color correction.